Audience measurement system for digital television

ABSTRACT

A television audience measurement system measures viewing of a television program viewed on digital television located in a statistically selected site by (i) retrieving an audience measurement data packet from a television set in order to identify the television program, (ii) detecting an audio code embedded in the television program in order to identify the television program, (iii) extracting an audio signature from the television program in order to identify the television program, (iv) identifying the television program through use of a software agent, and (v) selecting at least one of the retrieving means, the detecting means, the extracting means, and the software agent in order to identify the television program.

TECHNICAL FIELD OF THE INVENTION

[0001] The present invention relates to the measurement of audiences ofdigitally broadcast television programming and to the measurement ofusage of other information services provided to the audiences.

BACKGROUND OF THE INVENTION

[0002] Of the many approaches to measuring the usage of electronicentertainment equipment (commonly called “tuning” data), one approachinvolves the addition of an identifying code to a radio or televisionprogram, the distribution of the identifying code with the programthroughout the relevant broadcasting system, and the detection andinterpretation of the identifying code when the broadcast signal isviewed or heard in a statistically selected monitoring site. An exampleof-a system which implements this type of measurement approach may befound in the following patents: U.S. Pat. No. 5,481,294 to Thomas etal., who describe, inter alia, identifying codes added to the verticalblanking interval of an NTSC television broadcast; U.S. Pat. No.5,629,739 to Dougherty, who is particularly concerned with the additionof an identifying code to a low energy portion of the audio spectrum ofan NTSC signal; and U.S. Pat. No. 5,404,377 to Moses, who teaches anaudio encoding arrangement using signal masking to decrease theperceptibility of the identifying code. The disclosures of U.S. Pat.Nos. 5,481,294, 5,629,739, and 5,404,377 are herein incorporated byreference.

[0003] Another approach to measuring usage of electronic entertainmentequipment involves the extraction of a characteristic feature signature(or characteristic feature signature set) from the programming selectedfor viewing, and the storing of a time-stamped signature (or signatureset) in a memory for subsequent transmission to a central datacollection office where the signature (or signature set) is comparedwith corresponding reference signatures collected by the central officefrom known broadcasting sources. This approach is taught by Lert and Luin U.S. Pat. No. 4,677,466. The teaching of U.S. Pat. No. 4,677,466 isherein incorporated by reference.

[0004] Yet another approach to the measurement of the usage ofelectronic entertainment equipment has been that of comparing the viewedsignal (or some component or artifact thereof) with all the signalsavailable to a sampled house-hold at the time the measurement is made. Areview of apparatus and methods useful for this measurement approach isfound in the teachings of Thomas et al. in U.S. Pat. No. 5,629,739 andof Lu et al. in U.S. Pat. No. 5,594,934. The teachings in U.S. Pat. No.5,594,934 and in U.S. Pat. No. 5,629,739 are herein incorporated byreference.

[0005] Wheeler, et al., in U.S. application Ser. No. 08/786,270, filedon Jan. 22, 1997, teach a video signal source detection arrangementwhich determines the source of a video and/or audio signal beingdisplayed by a television receiver. The arrangement implements a varietyof tuning measurement approaches. For example, it can be used to read anidentifying ancillary code transmitted with television programming, toobtain characteristic feature signatures from television programming, orto obtain a signal that is matched with a contemporary reference signalobtained by a television-tuner controlled by measurement equipment. Thedisclosure of the aforementioned U.S. application Ser. No. 08/786,270 isherein incorporated by reference.

[0006] Chan, in U.S. application Ser. No. 08/654,309, filed on May 28,1996, teaches a sensor arrangement for non-intrusively obtaining arepresentation of video and synchronization signals from a televisionreceiver. The disclosure of the aforementioned U.S. application Ser. No.08/654,309 is herein incorporated by reference.

[0007] Changes in the methods of measuring the reception of televisionor radio programming are required because of a planned change-over fromanalog to digital broadcasting. In the U.S., the change-over isscheduled to be phased in by the year 2006, as documented in the FederalCommunication Commission's Mass Media Docket 87-268, with particularreference to the Fifth Report and Order, FCC 97-116, Apr. 3, 1997 and tothe Sixth Report and Order, FCC 97-115, Apr. 3, 1997. Some of thechanges, and their respective impacts on approaches used for measuringanalog broadcasts, include the following: (1) digital broadcasttechniques do not use a vertical blanking' interval and, therefore, theprogram tracking and identifying codes that are written in the verticalblanking interval of analog broadcast signals will not be transmitted;(2) multiple signal formats and associated multiple display formatshaving, among other features, differing height-to-width ratios anddifferent resolutions are allowed, so that existing video signalcorrelation methods used with analog broadcasts may essentially bedisabled because these methods depend on having the same pictorialfeature appear at corresponding places on the measured and referencedisplays; (3) a broadcaster can transmit as many as six programs(arrayed as a sequence of data packets, where each data packet islabeled as to which of the programs' data is carried therein) within anassigned 6 MHZ frequency band by trading off pictorial resolution for anincrease in the number of programs so that (i) digital signalcompression methods used to decrease the spectrum space required by aprogram destroy program identifying codes embedded in an original,high-resolution, program master, and (ii) determining which channel hasbeen tuned by a receiver does not uniquely identify a program beingviewed if more than one program is being transmitted in that channel;and, (4) data other than television broadcasting may be co-transmittedin the same channel and, in some cases, it is expected that the otherdata will be related in some manner to the co-transmitted programming sothat a viewer can interact with the TV programming (e.g., to obtain aprogram guide or detailed information on an advertised product, toautomatically switch to a desired program, or to take part in anaudience-participation program).

[0008] It may be noted that, although projected digital signalcompression and transmission methods destroy the types of video codesused in, and suggested for, the prior-generation analog broadcastingapproaches, it is expected that less change will be encountered withrespect to audio codes. That is, because the audio component of atelevision broadcast comprises much less information than does thevideo, there is much less to be gained by applying signal compressionmethods to audio. Nonetheless, it is expected that various audiocompression methods may be employed. These methods include those definedby the Dolby AC-3 or the European MUSICAN standards, or those defined bysome yet-to-be-defined standard operable within the relatively open andflexible MPEG arrangement. Although the coding approaches taught in theaforementioned U.S. Pat. No. 5,404,377 are known to survive the DolbyAC-3 and MUSICAN compression methods, compression methods other thanDolby AC-3 and MUSICAN may destroy these codes. Moreover, although thechange in technology may obviate the use of some of the codes, such-asthose described above, other codes (e.g., digital data packet codestelling a receiving site which of several interleaved programs isassociated with a given data packet) are expected to be broadcast bothwith television programming and with any co-transmitted data that arerelated to, and intended to be used in conjunction with, one or more ofthe programs.

[0009] In addition to changes in transmission equipment and signals,considerable changes are expected in reception equipment. Notable amongthese are both the use of set top signal converters (generally referredto as set top boxes) allowing a digital broadcast to be viewed on ananalog receiver, and a projected convergence of television receivers andcomputers. In the short term, for example, it is expected that digitalreceivers, configured as plug-in boards for personal computers, will beused to enable the display of digital television signals on thecomputers.

[0010] Another traditional measurement reflected in television audiencereporting has been the determination of who actually viewed the programthat was received by the equipment being monitored in a statisticallyselected monitoring site. A related measurement of interest is theidentification of users of on-line services, such as on-line servicesprovided over the Internet. Also, it is clear that measuring the membersof an audience in a statistically selected monitoring site who areinteractively using a television and/or accessing the Internet, and theinformation delivery apparatus used to deliver information to themembers, will continue to be important.

[0011] By far the most commonly used approach in making thesemeasurements is that of interrogation, wherein the viewer and/orInternet user is asked to identify himself or herself as a member of thetelevision audience or as an Internet user. In connection withtelevision viewing, this inquiry is usually done by means of anelectronic prompting and data input device (commonly referred to as aPeople-meter) associated with a monitored receiver in a statisticallyselected monitoring site. The member identification may also includeage, sex, and other demographic data. It is common to store both thedemographic data and the tuning data associated with each monitoredreceiver in the statistically selected monitoring site instore-and-forward equipment located within the monitoring site and tosubsequently forward these data to a central office computer via adirect call over the public switched telephone network, or via theInternet, on a daily basis.

[0012] Of particular interest in this area is the teaching of anaudience interrogation arrangement disclosed by McKenna et al., in U.S.Pat. No. 4,816,904. According to this arrangement, a prompting messageis displayed on a television screen overlaid on viewer selectedprogramming by mixing the prompting message with the video signal beingsent to the display. However, McKenna et al. do not teach a “tiling”arrangement where the prompting message is displayed on a portion of adisplay separated from the portion of the display devoted to televisionpicture.

[0013] Many arrangements have been proposed in the computer arts forreliably identifying a user so that only selected individuals can secureaccess to some of the data on a computer or computer system. The mostcommon of such arrangements is that of requiring the user to enter bothhis or her claimed identity along with a password. Other arrangementshave included the use of various biometric techniques, such as image orvoice recognition devices. Additionally, it is well known in theaudience measurement arts to use computer-based image recognition inorder to identify members of a viewing audience. Notable among teachingsin this area is that by Lu in U.S. Pat. No. 4,858,000. The teaching ofthis patent is herein incorporated by reference.

SUMMARY OF THE INVENTION

[0014] According to one aspect of the present invention, a televisionaudience measurement system identifies a viewer selected televisionprogram from among a plurality of television programs simultaneouslybroadcast in a broadcast channel. The television audience measurementsystem comprises a channel detector and a comparitor. The channeldetector detects the broadcast channel in which the plurality oftelevision programs is broadcast. The comparitor sequentially comparesan audio component of the plurality of television programs broadcast inthe detected broadcast channel with an audio signal of the viewerselected television program so as to identify the viewer selectedtelevision program.

[0015] According to another aspect of the present invention, atelevision audience measurement system is provided for digitaltelevision equipment that is disposed in a statistically selectedlocation. The television audience measurement system comprises asoftware agent and an interface and communication apparatus. Thesoftware agent is adapted to read, from a data packet contained indigital television programming, a datum identifying a televisionprogram, and the software agent is stored in memory associated with thedigital television equipment. The interface and communication apparatusis adapted to transmit the identification datum to a remotely locatedcentral office.

[0016] According to still another aspect of the present invention, atelevision audience measurement system identifies a viewer selectedtelevision program from among a plurality of television programsbroadcast as a time division multiplexed sequence of data packets in abroadcast channel. The viewer selected television program is displayedon a television display in a statistically selected location. Thetelevision audience measurement system comprises receiving means,acquiring means, recovering means, and comparing means. The receivingmeans receives the time division multiplexed sequence of data packets inthe broadcast channel. The acquiring means acquires an audio portion ofthe viewer selected television program. The recovering means recoversaudio components respectively corresponding to the television programscontained in the sequence of data packets. The comparing means comparesthe audio components to the audio portion in order to determine theviewer selected television program.

[0017] According to yet another aspect of the present invention, atelevision audience measurement system measures viewing of a televisionprogram viewed on digital television located in a statistically selectedsite and comprises detecting means, extracting means, a software agent,and selecting means. The detecting means detects an audio code embeddedin the television program in order to identify the television program.The extracting means extracts an audio signature from the televisionprogram in order to identify the television program. The software agentis arranged to identify the television program. The selecting meansselects at least one of the detecting means, the extracting means, andthe software agent in order to identify the television program.

[0018] According to a further aspect of the present invention, a methodis provided to identify a television program selected by a viewer from aset of television programs broadcast as multiplexed data packets in aviewer selected broadcast channel. The viewer selected televisionprogram is displayed on a display portion of an apparatus tuned to theviewer selected broadcast channel. The method comprises the followingsteps: a) determining the viewer selected broadcast channel; b)acquiring an audio portion of the viewer selected television program; c)selecting an audio component associated with one of the set oftelevision programs broadcast in the viewer selected broadcast channel;d) comparing the audio portion with the audio component in order todetermine whether the audio portion and the audio component match; e) ifthe audio portion and the audio component match, storing a tuning recordfrom at least one of the audio portion and the audio component; and, f)if the audio portion and the audio component do not match, selecting adifferent one of the television programs from the set of televisionprograms and repeating steps (c) through (f) until either a match isfound or the set of television programs is exhausted.

[0019] According to a still further aspect of the present invention, amethod is provided to identify a viewer selected television program fromamong a plurality of time overlapped television programs broadcast in aviewer selected broadcast channel. The viewer selected televisionprogram is displayed in a first window of a multi-window televisiondisplay. A data file, logically related to the viewer selectedtelevision program, is also broadcast in the viewer selected channel soas to be time overlapped with the viewer selected television program.The viewer selected television program and the data file containrespective labels indicative of the logical relation. Data from the datafile is displayed in a second window of the multi-window display. Themethod comprises the following steps: a) reading, from the data file,the respective data file label and an identifying datum; b) finding,from the logical relation, the television program label associated withthe respective data file label; and, c) storing a time-stamped recordcomprising the identifying datum.

[0020] According to yet a further aspect of the present invention, asoftware agent is stored in memory associated with digital televisionequipment. The software agent is arranged to acquire television audiencemeasurement data relative to the digital television equipment. Thesoftware agent comprises first, second, and third logging means. Thefirst logging means logs a television program identification datumidentifying a television program selected for viewing on the digitaltelevision equipment. The second logging means logs a co-transmittedidentification datum associated with data co-transmitted with thetelevision program selected for viewing on the digital televisionequipment. The third logging means logs an Internet identification datumassociated with an Internet task of the digital television equipment.

BRIEF DESCRIPTION OF THE DRAWING

[0021] These and other features and advantages of the present inventionwill become more apparent from a detailed consideration of the inventionwhen taken in conjunction with the drawings in which:

[0022]FIG. 1 is a schematic block diagram depicting typical digitaltelevision broadcasting and reception equipment;

[0023]FIG. 2 is a schematic block diagram depicting an audio-basedtuning measurement system which is usable with either digital or analogbroadcasting and which is in accordance with a first embodiment of thepresent invention;

[0024]FIG. 3 is a schematic block diagram depicting a tuning measurementsystem which employs a software agent resident in some of the monitoredreceiving equipment and which is in accordance with a second embodimentof the present invention;

[0025]FIG. 4 is a schematic block diagram depicting the use of digitalsource detection in accordance with a third embodiment of the presentinvention;

[0026]FIG. 5 is a schematic block diagram depicting the use of digitalsource detection in accordance with a fourth embodiment of the presentinvention;

[0027]FIG. 6 is a schematic block diagram depicting the use of dataports on various television viewing equipment in order to measuretelevision related data in accordance with a fifth embodiment of thepresent invention; and,

[0028]FIG. 7 is a flow chart depicting the operation of a software agentof the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0029] A typical digital television broadcasting environment, which isplanned during a transition period in which both digital and analogprograms are transmitted, may comprise some combination of the elementsdepicted in FIG. 1. Network programming, and other non-localprogramming, is provided to a local television station 10 from abroadcast station over a communication medium 12, such as a satellite,or terrestrial digital transmission lines, or the like. Localprogramming is also provided to the local television station 10 fromsome mixture of high definition television (HDTV) cameras 14 (which mayemploy a variety of formats), standard definition digital video cameras16 (which may employ a variey of resolutions and aspect ratios), andanalog sources comprising both analog video cameras 18 (e.g., thosecompatible with the prior generation NTSC broadcast standard), andcinema film sources 20 that have historically been converted to the NTSCstandard by means of well known telecine 22.

[0030] The signals from the analog video cameras 18 and from the cinemafilm sources 20 are input to an A/D converter 24 having an appropriatedigital output. The outputs from the A/D converter 24, from thecommunication medium 12, from the high definition television (HDTV)cameras 14, and/or from the standard definition digital video cameras 16are provided to a switching and routing apparatus 26. The switching androuting apparatus 26 passes selected ones of the signals provided to itthrough digital compression equipment 28 prior to being broadcast fromthe station's transmitter 30. The station's transmitter 30 may alsoinclude an antenna, a satellite dish, a cable modem, or the like.

[0031] In addition, a cable headend 31 (which is shown as a single cableprovider but which may represent a plurality of cable providers)receives signals over the air, from satellite distribution, or the like,and distributes the signals (with or without reformatting) by way ofregular cables, optical fibers, microwaves, or the like.

[0032] The digital transmission from the local television station 10and/or from the cable headend 31 may be directly received and displayedin a statistically selected monitoring site 32 by a digital televisionset 34. The digital transmission also may be directly received andconverted by a digital converter 36 (commonly configured as a set topbox or STB) into an analog output (e.g., in the NTSC format) for use byan analog television set 38 purchased by a viewer prior to thechange-over to digital programming. The digital transmission further maybe directly received and displayed by a personal computer 40 that has adigital receiver 42 providing a baseband video output signal that thepersonal computer 40 can show in one of the windows 44 defined on itsdisplay unit 46. The digital receiver 42, for example, could be a plugin board of the personal computer 40 or integrated on the motherboard.The personal computer also may have speakers 47, a keyboard 48, and amouse 50 or other pointing device known in the art.

[0033] Moreover, the digital transmission may be directly received andconverted by a digital converter 52 to a form useable by a digitaltelevision set 54. For example, the digital converter 52 may be used forpay-per-view television programming, video-on-demand televisionprogramming, satellite programs, and/or other services provided to thestatistically selected monitoring site 32. The digital transmission alsomay be directly received and converted by a digital converter 56 to aform useable by a simple monitor 58. In either of these two cases, thedigital television viewing can be measured by means of meteringtechnologies similar to the set top box and an analog televisionconfiguration described below.

[0034] In addition to the direct reception of the digital transmissionby apparatus dedicated solely to that purpose (e.g., the digitaltelevision set 34 and/or the personal computer 40), it is expected thata variety of yet-to-be-defined consumer electronic apparatuses will beconfigured Add that incorporate at least some of the features of both adigital or analog television receiver and a home computer.

[0035] A first embodiment 60 of the present invention, which measurestuning to digital television programming received in a statisticallyselected monitoring site 62, is depicted schematically in FIG. 2. Thefirst embodiment 60 measures such tuning by detecting programidentification codes and/or extracting signatures from the audio portionof the television signal to which a receiver is tuned. This audioportion is detected by a detector 64 in the case of a digital televisionset 66, and by a detector 68 in the case of an analog television set 70.The detector 64 may detect the audio portion of a program to which thedigital television set 66 is tuned by non-intrusively detecting thesound provided by a speaker 72 of the digital television set 66 (inwhich case the detector 64 may be a microphone) or by intrusivelydetecting the audio signal from a direct connection to the speaker, theaudio output terminals, or the audio processing circuitry of the digitaltelevision set 66. Similarly, the detector 68 may detect the audioportion of a program to which the analog television set 70 is tuned bynon-intrusively detecting the sound provided by a speaker 74 of thedigital television set 66 or by intrusively detecting the audio signalfrom a direct connection to the speaker, the audio output terminals, orthe audio processing circuitry of the analog television set 70. Theanalog television receiver 70 is provided with a digital converter 76,which may be similar to the digital converter 36 of FIG. 1.

[0036] The audio portion of a television program may also be detected bya detector 78 when the television program is played by a personalcomputer 80 on a monitor 82 and speakers 84. The detector 78 may detectthe audio portion of a program to which the personal computer 80 istuned by non-intrusively detecting the sound provided by the speakers 84or by intrusively detecting the audio signal from a direct connection tothe speaker, the audio output terminals, or the audio processingcircuitry of the personal computer 80.

[0037] As has been taught by Thomas et al., in U.S. Pat. No. 5,629,739,a representation of the audio portion picked up by the detector 64 canbe analyzed by a site unit (SU) 86 and/or a home unit (HU) 88 (which maybe provided with data storage and forwarding unit capability) within thestatistically selected monitoring site 62 in order to determine if theaudio portion from the digital television set 66 contains televisionprogram identification codes. Alternatively, the representation of theaudio portion picked up by the detector 64 can be analyzed at a remotelylocated data collection central office 90 in order to determine if theaudio portion from the digital television set 66 contains suchtelevision program identification codes. The remotely located datacollection central office 90 is schematically depicted as a computerbased data collection central office that interchanges data with thehome unit 88 over a communication network 92, such as a public switchedtelephone network, the Internet, or the like.

[0038] Similarly, a representation of the audio portion picked up by thedetector 68 can be analyzed by a site unit 94 and/or the home unit 88 inorder to determine if the audio portion from the analog television set70 contains television program-identifying codes, and a representationof the audio portion picked up by the detector 78 can be analyzed by asite unit 96 and/or the home unit 88 in order to determine if the audioportion from the personal computer 80 contains televisionprogram-identifying codes. Alternatively, the representation of theaudio portions picked up by the detectors 68 and 78 can be analyzed atthe remotely located data collection central office 90 in order todetermine if the audio portions from the analog television set 70 andthe personal computer 80 contain television program identificationcodes.

[0039] Moreover, a person identifier 98 may be provided in order toidentify the persons watching television programming on the digitaltelevision set 66. The person identifier 98 may be video camera, an IRcamera, or the like. When such equipment is available in thestatistically selected monitoring site 62, the site unit 86 may employknown head location and face recognition software (e.g., as taught by Luin U.S. Pat. No. 4,858,000) for the identification of the viewingpersons and for the collection of other demographic data. Similarly,person identifiers may be provided in order to identify the personswatching television programming on the analog television set 70 and onthe personal computer 80.

[0040] Instead of analyzing the representations of the audio portionspicked up by the detectors 64, 68, and 78 in order to determine if theaudio portions from the corresponding digital television set 66, analogtelevision set 70, and personal computer 80 contain televisionprogram-identifying codes, characteristic signatures can be extractedfrom these representations for subsequent comparison, either in thestatistically selected monitoring site 62 or in the central office 90,with previously collected reference signatures in order to identify thetelevision programs to which the digital television set 66, the analogtelevision set 70, and the personal computer 80 are tuned.

[0041] Programs played by VCRs on the digital television set 66, on theanalog television set 70, and on the personal computer 80 can beidentified using either the code or the signature technique. Thus, ifthe programs being played carry program identification codes that arepreserved through the process of recording and replay in the case oftime shifted viewing, or that are recorded in the programs stored ontapes to be rented or sold, the program identification codes can bedetected at the time that the programs are played so that the audiencesof the programs may be measured. However, if the programs being playedare non-encoded programs, or if the program identification codes arelost (e.g., during the recording process), then the embodiments of thepresent invention described herein can determine the programs beingviewed, when the tapes are played, by extracting characteristicsignatures from these programs and by matching the extracted signaturesto reference signatures stored in a reference signature database. Inthis case, searching can be optimized by the use of well known hashingtechniques.

[0042] Although the present invention may use either the code techniqueor the signature technique in order to measure audiences of programsplayed on digital television equipment, the present invention may useboth of these two techniques so that one of the techniques backs up theother in the event that the one technique is unable to identify a viewerselected television program. For example, signatures are extracted andmatched in order to identify a viewer selected television program onlyif identifying codes cannot be detected. As a further alternative, thepresent invention may use both of these two techniques simultaneously.Demographic data may also be collected in connection with the presentinvention.

[0043] It may be noted that, although the first embodiment 60 measuresthe great majority of the overall audience, it is not arranged tomeasure interactive features (e.g., co-transmitted advertising detail)that are expected to be part of the digital broadcasting environment.Therefore, an important feature of the following embodiment is that itmeasures interactive features.

[0044] Accordingly, a second embodiment 100 of the present invention isprovided as shown in FIG. 3. The second embodiment 100 measures tuningto digital television programming in connection with a statisticallyselected monitoring site 102. As will be apparent from the discussionbelow, this second embodiment 100 may include some or all of thefeatures of the first embodiment 60, as well as the new featuresdescribed below in connection with the second embodiment 100.

[0045] According to this second embodiment 100, tuning data, andpossibly demographic data, are collected from digital televisionreception equipment that includes processing equipment preferably havingadequate non-volatile or battery-backed memory available so that analgorithm, hereinafter referred to as a software agent, can be storedthereon. The digital television reception equipment of the secondembodiment 100 may include (i) a personal computer 104 which may besimilar to the personal computer 40 of FIG. 1, (ii) a digital converter106 which may be similar to the digital converter 36 of FIG. 1, (iii) ananalog television set 108 operating in response to the digital converter106, and (v) a digital television set 110.

[0046] One or more software agents of the second embodiment 100, asdisclosed in greater detail hereinafter, monitor operating tasks inorder to create time-stamped records containing tuning and/or other dataregarding operation of the particular pieces of equipment in which thesoftware agents are resident. Such monitored operating tasks may be, forexample, messages passed from a remote or other control operated by auser to a tuner or other program selector in order to select atelevision program being broadcast in a channel, messages passed from acontroller of processing equipment to a tuner in order to select atelevision program being broadcast in a channel, or the like. Althoughsoftware agents are expected to be able to effectively eavesdrop on datatraffic passing though their hosts' data ports, the use of softwareagents would not fully measure tuning at viewing sites unless each itemof television equipment at those sites has a respective software agentwithin it so that a complete set of operating tasks can be captured.

[0047] Accordingly, a software agent 112 is included in the digitalconverter 106. The software agent 112 monitors operating tasks in orderto create time-stamped records containing tuning and/or other dataregarding operation of the digital converter 106 in which the softwareagent 112 is resident. Thus, the software agent 112 detects, forexample, a message relating to selection of a television program beingbroadcast in a selected channel. From this detected message, theidentity of the selected television program may be determined. Thesoftware agent 112 is also arranged to eavesdrop on data traffic passingthough its host's input/output data port 114, and to communicate overthe input/output data port 114. Accordingly, programs to which theanalog television set 108 is tuned may be detected through use of thesoftware agent 112. The software agent 112 may be downloaded over acommunication medium 116 to the digital converter 106 by use of theinput/output data port 114. Alternatively, the software agent 112 may bea plug in, may be resident on a floppy disk, or the like.

[0048] Similarly, a software agent 118 is included in the digitaltelevision set 110. The software agent 118 monitors operating tasksassociated with the digital television set 110 in order to createtime-stamped records containing tuning and/or other data regardingoperation of the digital television set 110 in which the software agent118 is resident. The software agent 118 is also arranged to cooperatewith its host's input/output data port 120. A software agent 122 isincluded in the personal computer 104. The software agent 122 monitorsoperating tasks associated with the personal computer 104 in order tocreate time-stamped records containing tuning and/or other dataregarding operation of the personal computer 104 in which the softwareagent 122 is resident. The software agent 122 is also arranged tocooperate with its host's input/output data port 124.

[0049] Moreover, a person identifier 126 may be provided in order toidentify the persons watching television programming on the personalcomputer 104. As in the case of the person identifier 98, the personidentifier 126 may be video camera, an IR camera, or the like, and maybe provided with any or all of the viewing equipment shown in FIG. 4.

[0050] The software agents 112, 118, and 122 may be telecommunicationenabled so that television audience data from all the viewing sites inthe statistically selected monitoring site 102 can be communicated, viathe Internet, a public telephone system, or the like, to a locallylocated or remotely located intermediate data collector 128 and then toa remotely located central office 130 through a communication channel170. The intermediate data collector 128 can be flexibly located eitherat the same statistically selected monitoring site 102, or at a remotelylocated site 166, or at the central office 130. For this purpose, theinput/output data ports 114, 120, and 124 may be serial ports, parallelports, universal serial buses (USB), firewires (according to IEEE 1394),or the like. The data transfer may, for example, be structured as abackground task and take the form of a sequence of data packets, each ofwhich is e-mailed or otherwise communicated to the remotely locatedintermediate data collector 128 during pauses in a user's on linesessions or at any other time. The intermediate data collector 128, forexample, may be a computerized data collector or an Internet serviceprovider. Alternatively, the intermediate data collector 128 may be ahome unit, in which case, the intermediate data collector 128 may belocated within the statistically selected monitoring site 102 and mayitself be arranged to communicate tuning and demographic data via e-mailor other communication to the central office 130.

[0051] The software agent 122 additionally may monitor the personalcomputer 104 for interactive uses such as Internet usage (e.g., throughlogging of Universal Resource Locators, URLs), application softwareusage, and television viewing supported by the personal computer 104.Similarly, to the extent that the digital converter 106 and the digitaltelevision set 110 are web-enabled, the software agents 114 and 118 maymonitor interactive uses such as Internet usage (e.g., through loggingof URLs) and application software usage, as well as television viewing.

[0052] Because not all equipment that is encountered by a research firmdoing television audience research, at least during the early part ofthe digital television broadcasting era, will be compatible with thesoftware agent approach discussed above, the second embodiment 100 ofthe measurement system of the present invention may be arranged toinclude many or all of the features found in the first embodiment 60.

[0053] As described above, the first embodiment 60 measures the greatmajority of the overall audience by detecting the programs to whichreceivers are tuned by detecting program identification codes embeddedin the programs, and/or by extracting signatures of the programs beingviewed and by comparing the extracted signatures to referencesignatures. Moreover, the second embodiment 100 measures not only theoverall audience, but also interactive features provided with theprogramming. The embodiments described below are specifically arrangedto measure, in an efficient manner, the digital television viewing datawhere multiple television programs are present in a common digitaltransmission channel (e.g., of 6 MHZ), particularly where a digitaltelevision set does not have an accessible operating system with which asoftware agent can reside. The first embodiment 60 is also useful wherethere is no I/O interface on the television set so that there is no easyway to read data from the television set.

[0054]FIG. 4 depicts a third embodiment 200 of the invention in which astatistically selected monitoring site 202, during the transition periodwhen both analog and digital broadcasting occur, includes, for example,a first viewing site 204, a second viewing site 206, and a third viewingsite 208. The first viewing site 204 includes a digital converter 210connected to an analog television set 212. The audio portion of aprogram to which the analog television set 212 is tuned is detected by adetector 214. The detector 214 may detect this audio portion bynon-intrusively detecting the sound provided by a speaker 216 of theanalog television set 212 or by intrusively detecting the audio signalfrom a direct connection to the analog television set 212.

[0055] A probe 218 is provided to detect the intermediate frequencysignal of the digital converter 210. A signal processor 220 demodulatesthe detected intermediate frequency signal to the baseband bit streamcontained in the detected intermediate frequency signal, and thendemultiplexes the baseband bit stream to recover the audio portion ofthe baseband bit stream.

[0056] A comparitor 222 may be arranged to first determine whether thereis a program identification code in the audio detected by the detector214. If there is a program identification code in the audio detected bythe detector 214, this program identification code identifies theprogram to which the analog television set 212 is tuned and is,therefore, logged. If there is no program identification code in theaudio detected by the detector 214, the comparitor 222 then scans theaudio portion of the baseband bit stream provided by the signalprocessor 220 in order to detect the audio in a first program containedin the intermediate frequency signal detected by the probe 218. Thecomparitor 222 compares this audio to the audio data from the detector214. If there is a match, the program to which the digital converter 210is tuned is known from the information provided by the signal processor220. If there is no match, the comparitor 222 then scans the audioportion of the baseband bit stream provided by the signal processor 220in order to detect the audio in a second program contained in theintermediate frequency signal detected by the probe 218, and so on untilthe comparitor 222 detects a match. Once a match is found, the major andminor channel numbers of the matched channel are known, and thecorresponding program identifying ancillary code can be read from thecorresponding data packet.

[0057] The second viewing site 206 includes a digital television set224. The audio portion of a program to which the digital television set224 is tuned is detected by a detector 226. The detector 226 detectsthis audio portion by non-intrusively detecting the sound provided by aspeaker 228 of the digital television set 224 or by intrusivelydetecting the audio signal from a direct connection to the digitaltelevision set 224.

[0058] A probe 230 is provided to detect the intermediate frequencysignal of the digital television set 224. A signal processor 232operates similarly to the signal processor 220 in order to provide theaudio portion of the baseband bit stream in the intermediate frequencysignal detected by the probe 230. A comparitor 234 operates similarly tothe comparitor 222 in order to first determine whether there is aprogram identification code in the audio detected by the detector 226and, if there is not, to scan the audio portion of the baseband bitstream provided by the signal processor 232 until a match is found.

[0059] The third viewing site 208 includes a personal computer 236. Theaudio portion of a program to which the personal computer 236 is tunedis detected by a detector 238. The detector 238 detects this audioportion by non-intrusively detecting the sound provided by speakers 240of the personal computer 236 or by intrusively detecting the audiosignal from a direct connection to the personal computer 236.

[0060] A probe 242 is provided to detect the intermediate frequencysignal of the personal computer 236. A signal processor 244 operatessimilarly to the signal processor 220 in order to provide the audioportion of the baseband bit stream in the intermediate frequency signaldetected by the probe 242. A comparitor 246 operates similarly to thecomparitor 222 in order to first determine whether there is a programidentification code in the audio detected by the detector 238 and, ifthere is not, to scan the audio portion of the baseband bit streamprovided by the signal processor 244 until a match is found.

[0061] The comparitors 222, 234, and 246 provide their programidentification codes from the audio or from the data packet, and/ormatch information to a home unit 248, which may be provided with datastorage and forwarding unit capability, within the statisticallyselected monitoring site 202. A remotely located data collection centraloffice 250 is schematically depicted as a computer based data collectioncentral office that interchanges data with the home unit 248 over acommunication network 252, such as a public switched telephone network,the Internet, or the like. Moreover, a person identifier 254 may beprovided in order to identify the persons watching televisionprogramming on the digital television set 224. As in the case of theperson identifier 98, the person identifier 254 may be video camera, anIR camera, or the like, and may be provided for any or all of theviewing equipment shown in FIG. 4.

[0062] Alternatively, instead of using intermediate frequency probes forthe probes 218, 230, and 242, local oscillator probes may be used. Alocal oscillator probe may be employed to initially identify the channeltuned by its associated television receiver, which offers a significantspeed advantage in the measurement. That is, because channel detectionapparatus normally operates much faster than does audio signalcomparison apparatus, the program determination process can be carriedout in a hierarchical fashion in which the broadcast channel is firstdetermined from the local oscillator signal by the signal processors220, 232, and 244 (assuming that a program identification code is notfirst detected). Then, the comparitors 222, 234, and 246 sequentiallytune to the programs co-transmitted in the detected broadcast channeland compare the audio portion of the tuned program to the audio datafrom detectors 214, 226, and 238 in order to identify the program in thebroadcast channel. In this way, the comparitors 222, 234, and 246 arenot required to sequentially tune through all of the programs in all ofthe available channels in order to determine the tuned program.

[0063] A still further alternative is provided by a fourth embodiment300 of the invention shown in FIG. 5. The fourth embodiment 300 is usedin a statistically selected monitoring site 302 having a first viewingsite 304, a second viewing site 306, and a third viewing site 308. Thefirst viewing site 304 includes a digital converter 310 connected to ananalog television set 312. The audio portion of a program to which theanalog television set 312 is tuned is detected by a detector 314. Thedetector 314 may detect this audio portion by non-intrusively detectingthe sound provided by a speaker 316 of the analog television set 312 orby intrusively detecting the audio signal from a direct connection tothe audio processing circuitry of the analog television set 312.

[0064] A scanning receiver 318 is arranged to sequentially tune to eachof the programs carried in each broadcast channel to which the digitalconverter 310 may be tuned, and to provide an audio portion of each suchprogram to a comparitor 320. The comparitor 320 may be arranged to firstdetermine whether there is a program identification code in the audiodetected by the detector 314. If there is a program identification codein the audio detected by the detector 314, this program identificationcode is logged because it identifies the program to which the analogtelevision set 312 is tuned. If there is no program identification codein the audio detected by the detector 314, the comparitor 320 thencompares the audio portion of a first program provided by the scanningreceiver 318 to the audio data from the detector 314. If there is amatch, the program to which the digital converter 310 is tuned is knownfrom the information provided by the scanning receiver 318. Once a matchis found, the major and minor channel numbers of the matched channel areknown, and the corresponding program identifying ancillary code can beread from the corresponding data packet.

[0065] If there is no match, the comparitor 320 compares the audioportion of a second program provided by the scanning receiver 318 to theaudio data from the detector 314, and so on until the comparitor 320detects a match. A demographic input 322 may be used to providedemographic data to the comparitor 320 which couples this demographicdata, and time, with the program to which the digital converter 310 istuned as a tuning record. The demographic input 322, for example, may bethe well know Peoplemeter with which people currently identifythemselves manually when they are viewing a program displayed on atelevision set.

[0066] The second viewing site 306 includes a digital television set324. The audio portion of a program to which the digital television set324 is tuned is detected by a detector 326. The detector 326 detectsthis audio portion by non-intrusively detecting the sound provided by aspeaker 328 of the digital television set 324 or by intrusivelydetecting the audio signal from a direct connection to the digitaltelevision set 324.

[0067] A scanning receiver 330 is arranged to sequentially tune to eachof the programs carried in each broadcast channel to which the digitaltelevision set 324 may be tuned, and to provide an audio portion of eachsuch program to a comparitor 332. The comparitor 332 may be arranged tofirst determine whether there is a program identification code in theaudio detected by the detector 326. If there is a program identificationcode in the audio detected by the detector 326, this programidentification code is logged because it identifies the program to whichthe digital television set 324 is tuned. If there is no programidentification code in the audio detected by the detector 326, thecomparitor 332 then compares the audio portion of a first programprovided by the scanning receiver 330 to the audio data from thedetector 326. If there is a match, the program to which the digitaltelevision set 324 is tuned is known from the information provided bythe scanning receiver 330. If there is no match, the comparitor 332compares the audio portion of a second program provided by the scanningreceiver 330 to the audio data from the detector 326, and so on untilthe comparitor 332 detects a match.

[0068] Moreover, a person identifier 334 may be provided in order toidentify the persons watching television programming on the digitaltelevision set 224. As in the case of the person identifier 98, theperson identifier 254 may be video camera, an IR camera, or the like,and may be provided for any or all of the viewing equipment shown inFIG. 4. The comparitor 332 couples the identities of the personswatching the digital television set 324, and time, with the program towhich the digital television set 324 is tuned as a tuning record.

[0069] The third viewing site 308 includes a personal computer 336. Theaudio portion of a program to which the personal computer 336 is tunedis detected by a detector 338. The detector 338 detects this audioportion by non-intrusively detecting the sound provided by speakers 340of the personal computer 336 or by intrusively detecting the audiosignal from a direct connection to the personal computer 336.

[0070] A scanning receiver 342 is arranged to sequentially tune to eachof the programs carried in each broadcast channel to which the personalcomputer 336 may be tuned, and to provide an audio portion of each suchprogram to a comparitor 344. The comparitor 344 may be arranged to firstdetermine whether there is a program identification code in the audiodetected by the detector 338. If there is a program identification codein the audio detected by the detector 338, this program identificationcode is logged because it identifies the program to which the personalcomputer 336 is tuned. If there is no program identification code in theaudio detected by the detector 338, the comparitor 344 then compares theaudio portion of a first program provided by the scanning receiver 342to the audio data from the detector 338. If there is a match, theprogram to which the personal computer 336 is tuned is known from themajor and minor channel numbers indicated by the match. If there is nomatch, the comparitor 344 compares the audio portion of a second programprovided by the scanning receiver 342 to the audio data from thedetector 338, and so on until the comparitor 344 detects a match. Thecomparitor 344 couples time with the program to which the personalcomputer 336 is tuned as a tuning record.

[0071] The comparitors 222, 234, and 246 provide their tuning records toa home unit 346, which may be provided with data storage and forwardingunit capability, within the statistically selected monitoring site 302.A remotely located data collection central office 348 is schematicallydepicted as a computer based data collection central office thatinterchanges data with the home unit 346 over a communication network350, such as a public switched telephone network, the Internet, or thelike.

[0072] It is likely that many, if not all, digital television sets,digital converters, and like digital equipment will have data ports thatwill provide output viewer-selected program information for use inconsumer devices connected to the data ports. In connection with digitaltelevisions, a data port is likely to provide, for example, the fullmajor channel ATSC bitstream and information on virtual channels thatmay include analog or digital major channel selection, program (minorchannel) selection, physical source (TV, VCR, DVD, etc.) identification,and the like. In connection with digital converters (i.e., set topboxes), a data port is likely to provide, for example, the full ATSCbitstream and information on virtual channels that may include analog ordigital major channel selection, program (minor channel) selection,vertical blanking interval data reinserted on the correct line and fieldof analog signals, and the like.

[0073] Accordingly, a fifth embodiment 400 of the present invention isprovided as shown in FIG. 6. The fifth embodiment 400 measures tuning todigital television programming in connection with a statisticallyselected monitoring site 402. The fifth embodiment 400 may include someor all of the features of the previously described embodiments.

[0074] The fifth embodiment 400 permits tuning data, and possiblydemographic data, to be collected from digital television receptionequipment that includes (i) a personal computer 404, (ii) a digitalconverter 406, (iii) an analog television set 408 operating in responseto the digital converter 406, and (v) a digital television set 410.

[0075] The digital converter 406 has a data port 412. A site unit 414reads and stores the full ATSC bitstream, the analog or digital majorchannel selection, the program selection, vertical blanking intervaldata, and/or the like data available at the data port 412 in order todetermine the program selected for viewing on the analog television set408 as well as any other desired and available information. Moreover,the site unit 414 may be arranged to collect demographic data related tothe viewers in the audience of the program selected for viewing on theanalog television set 408. For example, the site unit 414 may collectdemographic data from a person identifier, a Peoplemeter, or the like,as discussed above. All such data may be uploaded over a communicationmedium 416, which may be a wire, a power line, and IR link, an RF modem,or the like.

[0076] Similarly, the digital television set 410 has a data port 418. Asite unit 420 reads and stores the full major channel ATSC bitstream,the major channel selection, the program selection, physical sourceidentification, and the like data available at the data port 418 inorder to determine the program selected for viewing on the digitaltelevision set 410 as well as any other desired and availableinformation. Moreover, the site unit 420 may be arranged to collectdemographic data related to the viewers in the audience of the programselected for viewing on the digital television set 408. For example, thesite unit 420 may collect demographic data from a person identifier, aPeoplemeter, or the like, as discussed above. All such data may beuploaded over a communication medium 421.

[0077] Also, the personal computer 404 has a data port 422. A site unit424 reads and stores the full major channel ATSC bitstream, the majorchannel selection, the program selection, and the like data available atthe data port 422 in order to determine the program selected for viewingon the personal computer 404 as well as any other desired and availableinformation. Moreover, a person identifier 426 may be provided in orderto identify the persons watching television programming on the personalcomputer 404. As in the case of the person identifier 98, the personidentifier 426 may be video camera, an IR camera, or the like, and maybe provided with any or all of the viewing equipment shown in FIG. 6.All program and demographic data may be uploaded over a communicationmedium 427.

[0078] The site units 414, 420, and 424 may be telecommunication enabledso that television audience data from all the viewing sites in thestatistically selected monitoring site 402 can be communicated, via theInternet, a public telephone system, or the like, to a locally locatedor remotely located intermediate data collector 428 and then to aremotely located central office 430 through a communication channel 470.The intermediate data collector 428 can be flexibly located either atthe same statistically selected monitoring site 402, or at a remotelylocated site 466, or at the central office 430. For this purpose, thesite units 414, 420, and 424 may be provided with serial ports, parallelports, universal serial buses (USE), firewires (according to IEEE 1394),or the like. The intermediate data collector 428, for example, may be acomputerized lo data collector or an Internet service provider.Alternatively, the intermediate data collector 428 may be a home unit,in which case, the intermediate data collector 428 may be located withinthe statistically selected monitoring site 402 and may itself bearranged to communicate tuning and demographic data via e-mail or othercommunication to the central office 430.

[0079] A software agent 500 is shown in FIG. 7. The software agent 500can be used for any of the software agents 112, 118, and 122. As shownat a block 502 of the software agent 500, the software agent 500 copiesthe current operating task. If the current operating task uses a packetof television programming as determined at a block 504, the softwareagent 500 at a block 506 determines whether the data packet has adecodable packet label including a decodable program identificationcode, name, or other indicia. This program identification data packet isexpected to be a feature of digital television programming, particularlyif multiple programs are packed simultaneously as minor channels in abroadcast channel in a digital broadcasting environment, and is expectedto identify the program tuned by the monitored television.

[0080] If the data packet does not have a decodable packet label, thesoftware agent 500 at a block 508 causes the ON state of the monitoredviewing equipment to be logged, provided that this ON state is a newstatus for the monitored viewing equipment and the monitored viewingequipment is ON. The balance of the measurement system (e.g., the audiosignal methods employed by any of the other embodiments 60, 200, and 300described above) is relied on in order to identify the tuning. Programflow then returns to the block 502 to await the next operating task.

[0081] On the other hand, if the data packet does have a decodablepacket label, the software agent 500 determines at a block 510 whetherthe packet label from the data packet of the current operating task isthe same as the packet label previously determined by the software agent500. If so, there is no need to store the packet label unless it ismerely indicative of continued viewing to an already identifiedtelevision program. Accordingly, program flow returns to the block 502to await the next operating task. However, if the packet label from thedata packet of the current operating system task is not the same as thepacket label previously determined at the block 506, the software agent500 at a block 512 logs the identification of the television program ascontained in the program label and also logs the time. Program flow thenreturns to the block 502 to await the next operating task.

[0082] If the current operating task does not use a packet of televisionprogramming as determined at a block 504, the software agent 500 at ablock 514 determines whether the current operating task is a command toother monitored equipment (e.g., a command to the digital televisionreceiver 110 to tune a different channel and to select a particular oneof the N programs being broadcast in that channel). If the currentoperating task is a command to other monitored equipment, the softwareagent 500 at a block 516 logs as much detail as is available (e.g., thecommand itself, the identity of the issuer of the command, and theidentity of the recipient of the command) in memory, and program flowreturns to the block 502.

[0083] If the current operating task is not a command to other monitoredequipment, the software agent 500 at a block 518 determines ifco-transmitted data (i.e., data that is related to one of the televisionprograms being broadcast in a channel and that is transmitted during thesame time interval as the program) has been selected by a viewer.Co-transmitted data may comprise a guide to other available televisionprogramming, catalog-like details on products being advertised on theco-transmitted program, and the like. Some of these proposed uses ofco-transmitted data will be configured so that one can infer whatprogram is being viewed from a URL or other label that logically linksthe co-transmitted data to the television program. Accordingly, thesoftware agent 500 at a block 520 searches the header portion of a taskfor such URL or other label, and logs such URL or other label that isfound. Thereafter, program flow returns to the block 502.

[0084] If co-transmitted data has not been selected by a viewer, thesoftware agent 500 at a block 522 determines if the user of theequipment has directed his or her browser, or other communicationsoftware, to access an internet page (assuming that the monitoredviewing equipment being monitored is configured so that it has access tothe Internet or other on-line service). If the user of the equipment hasdirected his or her browser, or other communication software, to accessan internet page, the software agent 500 at a block 524 searches for aURL or other label, and logs such URL or other label that is found.Thereafter, program flow returns to the block 502. If the user of theequipment has not directed his or her browser, or other communicationsoftware, to access an internet page, program flow returns to the block502.

[0085] As an example of the use of a software agent as describe above,consider the case of a person in a statistically selected monitoringsite who is viewing a digital television program and who sees aninteresting product being advertised. It is expected that the personwill have the capability to request more data on that product by meansof a simple mouse-click operation. This data, for example, will eitherbe a subset of the data providing the displayed advertisement, or willbe co-transmitted subsequently in the signal of the television programthat the person has selected for viewing. This data would then bedisplayed in a separate window on the multi-windowed display being usedfor television viewing. An additional set of data could then be obtainedby clicking on a hypertext link element of the initially displayed data.This action would result in downloading the desired additional data overthe Internet. A software agent installed in this viewing equipment couldthus log not only what program was being viewed, but could also supplyadditional marketing research data on the consumer's use of in-programadvertising.

[0086] In addition to providing a device for monitoring tuning activity,the software agent of the present invention can be used to collectdemographic data relative to the monitored viewing. In a manual approachto the collection of demographic data, the software agent could generatea status and prompting display in one window of the viewing equipmentrequiring a viewer or user to input the appropriate demographic data.This status and prompting display window could have an appearancesimilar to that provided by the peoplemeter hardware widely used incontemporary audience measurements and that displays a status reminderlight for each occupant of a statistically selected monitoring site.However, in the comparable windowed display arrangement, the name ofeach occupant could be shown along with a status indicator.

[0087] If the software agent is used with a personal computer that canreceive digital television programming, a variety of more nearlyautomatic and less intrusive approaches (from the television viewer'sperspective) could be used to collect persons data and update a tiledstatus display. For example, it is well known to identify an individualuser of a manual input device (such as a keyboard or a mouse) of acomputer by the characteristic rates of data entry and pauses betweendata entries. In a television audience measurement, in which theoccupant population to be monitored is very small, measurements of thissort should be expected to be highly accurate. Moreover, an increasingnumber of personal computers have video cameras associated with them, sothat the user of the computer can engage in video conferences bytelephone. When such equipment is available in a statistically selectedmonitoring site, a software agent comprising known head location andface recognition software (e.g., as taught by Lu in U.S. Pat. No.4,858,000) can be employed for the identification persons and thecollection of demographic data.

[0088] Certain modifications of the present invention have beendiscussed above. Other modifications will occur to those practicing inthe art of the present invention. For example, the software agents 112,118, and 122, as described above, monitor operating tasks. The operatingtasks may be operating system events of the operating systems used bythe equipment in which the software agent is resident. Indeed, thesoftware agents 112, 118, and 122 may monitor any function of themonitored equipment as long as the desired data is collected.

[0089] Also, a statistically selected monitoring site according to eachof the embodiments of the invention, as described above, includescertain viewing equipment as well as certain monitoring equipment.However, it should be understood that a statistically selectedmonitoring site according to the present invention may include anycombination of the equipment incorporated in the embodiments describedabove, as well as other and/or different equipment.

[0090] Moreover, the software agent 500 described above can be genericso that it can be used with any piece of equipment with which it iscompatible, or it may be specialized for each particular apparatus inwhich it is used. Also, the software agent 500 may be arranged to detectthe all window activities conducted by an audience.

[0091] Furthermore, although not shown in FIGS. 2-5, the digitaltelevision reception equipment of the embodiments shown therein mayfurther include auxiliary digital television equipment such as a VCR, adigital video disk player, a video game, or other entertainment systems.

[0092] Additionally, as described above, program identification codesare detected in order to identify television programs. However,television programs may be identified by any indicia such as programnames. Moreover, an internet page may be identified by its address or byan identification code or name or label or, in the case of anadvertisement on the Internet, by its banner. Therefore, any indiciafrom which a television program or content associated with a televisionprogram may be identified is referred to herein generically as anidentification datum.

[0093] Moreover, the home units 248 and/or 346 may be referred to hereinan intermediate data collectors.

[0094] Accordingly, the description of the present invention is to beconstrued as illustrative only and is for the purpose of teaching thoseskilled in the art the best mode of carrying out the invention. Thedetails may be varied substantially without departing from the spirit ofthe invention, and the exclusive use of all modifications which arewithin the scope of the appended claims is reserved.

We claim:
 1. A television audience measurement system for identifying aviewer selected television program from among a plurality of televisionprograms simultaneously broadcast in a broadcast channel, the televisionaudience measurement system comprising: a channel detector that detectsthe broadcast channel in which the plurality of television programs isbroadcast; and, a comparitor that sequentially compares an audiocomponent of the plurality of television programs broadcast in thedetected broadcast channel with an audio signal of the viewer selectedtelevision program so as to identify the viewer selected televisionprogram.
 2. The television audience measurement system of claim 1wherein the audio signal is detected by a microphone disposed adjacent atelevision display.
 3. The television audience measurement system ofclaim 1 wherein the audio signal is detected by an intrusive connectioninto equipment related to television viewing.
 4. The television audiencemeasurement system of claim 1 further comprising a programidentification datum detector that detects, from the viewer selectedtelevision program, a program identification datum indicative of anidentity of the viewer selected television program.
 5. The televisionaudience measurement system of claim 4 wherein the programidentification datum detector comprises a software agent resident inequipment related to television viewing.
 6. The television audiencemeasurement system of claim 5 wherein the software agent is arranged tosearch for the program identification datum.
 7. The television audiencemeasurement system of claim 4 wherein the audio signal is detected by amicrophone disposed adjacent a television display.
 8. The televisionaudience measurement system of claim 4 wherein the audio signal isdetected by an intrusive connection into equipment related to televisionviewing.
 9. The television audience measurement system of claim 4wherein the comparitor is arranged to sequentially compare the audiocomponent of the plurality of television programs broadcast in thedetected broadcast channel with the audio signal of the viewer selectedtelevision program so as to identify the viewer selected televisionprogram if the program identification datum detector is unable to detecta program identification datum indicative of an identity of the viewerselected television program.
 10. The television audience measurementsystem of claim 1 further comprising a software agent arranged todetect, from the viewer selected television program, a data elementindicative of an identity of the viewer selected television program. 11.The television audience measurement system of claim 10 wherein thesoftware agent is arranged to search for a program identification datumindicative of an identity of the viewer selected television program. 12.The television audience measurement system of claim 4 wherein thecomparitor is arranged to sequentially compare the audio component ofthe plurality of television programs broadcast in the detected broadcastchannel with the audio signal of the viewer selected television programso as to identify the viewer selected television program if the programidentification datum detector is unable to detect a programidentification datum indicative of an identity of the viewer selectedtelevision program.
 13. A television audience measurement system fordigital television equipment, wherein the digital television equipmentis disposed in a statistically selected location, the televisionaudience measurement system comprising: a software agent adapted toread, from a data packet contained in digital television programming, adatum identifying a television program, wherein the software agent isstored in memory associated with the digital television equipment; aninterface and communication apparatus adapted to transmit theidentification datum to a remotely located central office.
 14. Thetelevision audience measurement system of claim 13 wherein the digitaltelevision equipment comprises a receiver having a tuner, amicroprocessor, memory, an operating system, and a video display unit.15. The television audience measurement system of claim 13 wherein thedigital television equipment is a set top box providing an analogtelevision signal to an analog receiver.
 16. The television audiencemeasurement system of claim 13 wherein the digital television equipmentcomprises a set top box providing a digital television signal to adigital receiver.
 17. The television audience measurement system ofclaim 13 wherein the digital television equipment comprises a set topbox and a monitor.
 18. The television audience measurement system ofclaim 13 wherein the digital television equipment comprises a personalcomputer provided with a television receiver.
 19. The televisionaudience measurement system of claim 13 wherein the digital televisionequipment includes a VCR.
 20. The television audience measurement systemof claim 13 wherein the digital television equipment includes a digitalversatile disk player.
 21. The television audience measurement system ofclaim 13 further comprising a person identification apparatus.
 22. Thetelevision audience measurement system of claim 13 wherein theidentification datum is a program identification code of a televisionprogram.
 23. The television audience measurement system of claim 13wherein the identification datum comprises a program name.
 24. Thetelevision audience measurement system of claim 13 wherein theidentification datum comprises an address of an Internet page.
 25. Thetelevision audience measurement system of claim 13 wherein theidentification datum comprises an identification code of an Internetpage.
 26. The television audience measurement system of claim 13 whereinthe identification datum comprises a banner of material viewed by anaudience.
 27. The television audience measurement system of claim 13wherein the identification datum comprises a signature extracted from atelevision program viewed on the digital television equipment.
 28. Thetelevision audience measurement system of claim 13 wherein the softwareagent is arranged to detect window activities conducted by an audience.29. The television audience measurement system of claim 13 wherein theinterface and communication apparatus includes a serial port.
 30. Thetelevision audience measurement system of claim 13 wherein the interfaceand communication apparatus includes a parallel port.
 31. The televisionaudience measurement system of claim 13 wherein the interface andcommunication apparatus includes a universal serial bus.
 32. Thetelevision audience measurement system of claim 13 wherein the interfaceand communication apparatus includes a firewire.
 33. The televisionaudience measurement system of claim 13 wherein the interface andcommunication apparatus is arranged to send the identification datum toan Internet service provider via the Internet.
 34. The televisionaudience measurement system of claim 13 wherein the interface andcommunication apparatus includes an intermediate data collector.
 35. Thetelevision audience measurement system of claim 34 wherein theintermediate data collector includes a store and forward device, andwherein the store and forward device is arranged to send theidentification datum to the central office via a telephone line.
 36. Thetelevision audience measurement system of claim 34 wherein theintermediate data collector is an Internet service provider.
 37. Thetelevision audience measurement system of claim 34 wherein theintermediate data collector is a data collection facility located in thecentral office.
 38. The television audience measurement system of claim13 wherein the software agent is a software agent downloaded to thememory associated with the digital television equipment.
 39. Thetelevision audience measurement system of claim 13 wherein the softwareagent is a plug in software agent of the digital television equipment.40. The television audience measurement system of claim 13 wherein thesoftware agent is a floppy disk software agent of the digital televisionequipment.
 41. A television audience measurement system for identifyinga viewer selected television program from among a plurality oftelevision programs broadcast as a time division multiplexed sequence ofdata packets in a broadcast channel, the viewer selected televisionprogram being displayed on a television display in a statisticallyselected location, the television audience measurement systemcomprising: receiving means for receiving the time division multiplexedsequence of data packets in the broadcast channel; acquiring means foracquiring an audio portion of the viewer selected television program;recovering means for recovering audio components respectivelycorresponding to the television programs contained in the sequence ofdata packets; and, comparing means for comparing the audio components tothe audio portion in order to determine the viewer selected televisionprogram.
 42. The television audience measurement system of claim 41wherein the acquiring means comprises a sensor arranged to acquire arepresentation of a speaker signal from a speaker associated with thetelevision display.
 43. The television audience measurement system ofclaim 41 wherein the acquiring means comprises a connection to audioprocessing circuitry associated with the television display.
 44. Thetelevision audience measurement system of claim 41 wherein the receivingmeans comprises; an intermediate frequency probe arranged to acquire anintermediate frequency signal from a viewer controlled tuner associatedwith the television display; and, demodulating means for demodulatingthe intermediate frequency signal in order to receive the data packets.45. The television audience measurement system of claim 41 wherein thereceiving means comprises: a local oscillator frequency probe arrangedto pick up a local oscillator frequency signal from the televisiondisplay; means for identifying the broadcast channel from the localoscillator signal; wherein the recovering means recovers the audiocomponents from television programs contained in the identifiedbroadcast channel; and, wherein the comparing means compares the audiocomponents to the representation of the speaker signal in order todetermine the viewer selected television program.
 46. The televisionaudience measurement system of claim 41 wherein the receiving meanscomprises a scanning receiver arranged to scan each of a plurality ofbroadcast channels and to receive a corresponding plurality of timedivision multiplexed television programs from each of the plurality ofbroadcast channels.
 47. The television audience measurement system ofclaim 41 further comprising identifying means for identifying persons inan audience of the viewer selected television program.
 48. A televisionaudience measurement system for measuring viewing of a televisionprogram viewed on digital television located in a statistically selectedsite comprising: detecting means for detecting an audio code embedded inthe television program in order to identify the television program;extracting means for extracting an audio signature from the televisionprogram in order to identify the televsion program; a software agentarranged to identify the television program; and, selecting means forselecting at least one of the detecting means, the extracting means, andthe software agent in order to identify the television program.
 49. Thetelevision audience measurement system of claim 48 further comprisingretrieving means for retrieving an audience measurement data packet froma television set in order to identify the television program, whereinthe selecting means selects at least one of the retrieving means, thedetecting means, the extracting means, and the software agent in orderto identify the television program.
 50. A method of identifying atelevision program selected by a viewer from a set of televisionprograms broadcast as multiplexed data packets in a viewer selectedbroadcast channel, the viewer selected television program beingdisplayed on a display portion of an apparatus tuned to the viewerselected broadcast channel, the method comprising the steps of: a)determining the viewer selected broadcast channel; b) acquiring an audioportion of the viewer selected television program; c) selecting an audiocomponent associated with one of the set of television programsbroadcast in the viewer selected broadcast channel; d) comparing theaudio portion with the audio component in order to determine whether theaudio portion and the audio component match; e) if the audio portion andthe audio component match, storing a tuning record from at least one ofthe audio portion and the audio component; and, f) if the audio portionand the audio component do not match, repeating steps (c) through (f)until either a match is found or the set of television programs isexhausted.
 51. The method of claim 50 wherein step a) comprises the stepof determining the viewer selected broadcast channel by use of a channeldetector associated with the apparatus.
 52. The method of claim 51wherein step b) comprises the step of acquiring the audio portion of theviewer selected television program by use of an audio probe adjacent theapparatus.
 53. The method of claim 52 wherein step c) comprises the stepof selecting the audio component by use of a digital tuner that is not aportion of the apparatus.
 54. The method of claim 50 wherein step b)comprises the step of acquiring the audio portion of the viewer selectedtelevision program by use of an audio probe adjacent the apparatus. 55.The method of claim 54 wherein step c) comprises the step of selectingthe audio component by use of a digital tuner that is not a portion ofthe apparatus.
 56. The method of claim 50 wherein step c) comprises thestep of selecting the audio component by use of a digital tuner that isnot a portion of the apparatus.
 57. A method of identifying a viewerselected television program from among a plurality of time overlappedtelevision programs broadcast in a viewer selected broadcast channel,wherein the viewer selected television program is displayed in a firstwindow of a multi-window television display, wherein a file is alsobroadcast in the viewer selected channel so as to be time overlappedwith the viewer selected television program, wherein the viewer selectedtelevision program and the data file contain respective labels, whereinmaterial from the file is displayed in a second window of themulti-window display, and wherein the method comprises the steps of: a)reading, from the file, the respective file label and an identifyingdatum; b) finding the television program label associated with therespective file label; and, c) storing a time-stamped record comprisingthe identifying datum.
 58. The method of claim 57 wherein the file is adata file.
 59. The method of claim 57 wherein the file is an image file.60. The method of claim 57 wherein the file is logically related to theviewer selected television program, wherein the television program labeland the file label are indicative of the logical relation, and whereinstep comprises the step of finding, from the logical relation, thetelevision program label associated with the respective data file label.61. A software agent stored in memory associated with digital televisionequipment, wherein the software agent is arranged to acquire televisionaudience measurement data relative to the digital television equipment,the software agent comprising: first logging means for logging atelevision program identification datum identifying a television programselected for viewing on the digital television equipment; second loggingmeans for logging an identification datum associated with datacorresponding to the television program selected for viewing on thedigital television equipment; and, third logging means for logging anInternet identification datum associated with an Internet task of thedigital television equipment.
 62. An apparatus for identifying a viewerselected television program from among a plurality of time overlappedtelevision programs broadcast in a viewer selected broadcast channel andreceived by digital television program reception equipment, wherein thedigital television program reception equipment has a data port, theapparatus comprising: reading means connected to the data port forreading program identifying data from among data provided on the dataport; and, storing means for storing the program identifying data. 63.The apparatus of claim 62 wherein the digital television programreception equipment is a digital converter.
 64. The apparatus of claim62 wherein the digital television program reception equipment is apersonal computer.
 65. The apparatus of claim 62 wherein the digitaltelevision program reception equipment is a digital television set. 66.An apparatus for identifying a viewer selected television program fromamong a plurality of time overlapped television programs broadcast in aviewer selected broadcast channel and received by digital televisionprogram reception equipment, wherein the digital television programreception equipment has a data port, the apparatus comprising: readingmeans connected to the data port for reading program identifying datafrom among data provided on the data port; and, communicating means forcommunicating the program identifying data to a remote point.
 67. Theapparatus of claim 66 wherein the digital television program receptionequipment is a digital converter.
 68. The apparatus of claim 66 whereinthe digital television program reception equipment is a personalcomputer.
 69. The apparatus of claim 66 wherein the digital televisionprogram reception equipment is a digital television set.